Web Crawler

Web Crawler

美 [ˈweb ˌkrɔlər] 　英 [ˈweb ˌkrɔːlə(r)]

• n.网路蜘蛛
• 网络网络爬虫；网络爬虫或网络蜘蛛；网路爬虫

词形变化

复数：web crawlers  

英汉解释

n.

1. 网路蜘蛛

英英解释

n.	1. a program used for searching through websites on the Internet for documents containing a particular word, phrase, or subject

例句

If you avoid non-local links and dig down only into local Web pages, you provide a Web crawler for a single Web site, as shown in Listing 7.

如果能够避免非本地的链接而只访问本地Web页面，就可以为这个单一Web站点提供Web爬虫了，如清单7所示。

The sitemap files can then be used to indicate to a web crawler how such pages can be found.

在Sitemap文件可以用来显示的Web抓取工具如何对这些网页可以找到。

On this basis, the author proposed the implementation of this paper-content detection based on the web crawler technology.

在此基础上提出本文所选用的实现方案——基于网络爬虫技术的内容探测技术。

Each search engine has its own automated program called a "web spider" or "web crawler" that crawls the web.

每个搜索引擎都有自己爬行网页的自动化程序，叫做“网络蜘蛛（webspider）”或“网络爬虫（webcrawler）”。

The CollectUrls Web crawler program takes advantage of a fixed-size thread pool.

CollectUrlsWebcrawler程序利用一个固定大小的线程池。

Generally, search engine is consisting of web crawler, index construction, retrieval and user interface.

一般来说，搜索引擎都由搜索器、索引器、检索器和用户接口四个部分组成。

First, all frequent phrases that Spock extracts using its semantic technology via its web crawler become tags.

首先，Spock使用语义学技术、通过网络爬虫系统采集到的所有常用短语都会成为标签。

This information helps the Web crawler determine what the set of pages is and when to crawl them.

这一信息能帮助web爬虫程序决定要爬行哪些页面以及爬行的时间。

Gets a value indicating whether the browser is a search engine Web crawler.

获取一个值，该值指示浏览器是否为Web爬行遍历搜索引擎。

And real project experience in the areas of web crawler, search engines and information retrieval is a plus.

因此我们优先录取在网络爬虫开发、搜索引擎开发和信息检索等方面具有深厚技术基础的候选人。

If using a web crawler, this could be done by publishing a web page on a local site running on IBM HTTP Server.

如果使用的是web爬行器，那么可以通过在IBMHTTPServer上运行的本地站点中发布一个web页面实现。

A MapReduce application or a web crawler application fits perfectly with this model.

典型的如MapReduce框架，或者一个webcrawler应用都很适合这个模型。

A lot of bad web crawler, often complicated by dozens of hundreds of requests to grab cycle is repeated.

很多糟糕的网络爬虫，经常并发几十上百个请求循环重复抓取。

We designed a deep web crawler base on the most efficient queries.

本文提出一种基于最优查询词的深度网络爬虫。

The focused web crawler has been became a researching hotspot in technology domain of searching engine.

聚焦网络爬虫已经成为搜索引擎技术领域的一个研究热点，对于专业领域的搜索产生重要的作用。

Finally, it is illustrated to the future of Web crawler search engine research trends.

最后对搜索引擎未来网络爬虫研究趋势做了说明。

Figure 4 shows a snapshot of a Web crawler that is processing the whitehouse. gov site.

图4显示了正在处理whitehouse.gov站点的Web搜寻程序的快照。

Firstly, we introduce the main function of the web crawler and the present condition of technology of web crawler;

论文首先介绍了网络爬虫所起的作用以及网络爬虫技术的发展现状；

Design of A Multi-strategy Topic Specific Web Crawler

多策略的主题集中式万维网爬虫设计

The Implement and Design of a Prototype Web Crawler

一种主题型Web爬行器的设计和实现

Research and Realization of Intelligent Focused Web Crawler

基于主题的智能Web信息采集系统的研究与实现

Improved algorithm about topic web crawler's search strategy

一种改进的主题网络蜘蛛搜索算法

One Solution About Topic Web Crawler's Greedy Search Strategy

主题网络蜘蛛搜索策略贪婪性解决方法

Analysis on Web Crawler's Searching Strategies

简析搜索引擎中网络爬虫的搜索策略

Analysis and Design of Distributed Web Crawler

分布式Web信息采集系统的研究与设计

Design and implementation of focused Web crawler based on semantic analysis

基于语义分析的主题信息采集系统的设计与实现

Research and Design of Customized Web Crawler Based on Web Page Segmentation

基于网页分块的个性化信息采集的研究与设计

On the research and design of deep web crawler

爬虫研究与设计